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DECLARATION OF FACT UNDER 37 C.F.R, S1.131 



We, Renee M. Kovales, Edith H. Stern, and Barry E. Willner, hereby declare the following: 
1) We are co-inventors of the invention described and claimed in U. S. Patent Application 
Number 10/632,177 (hereinafter, "the Subject Application"), entitled "Selectable Audio and 
Mixed Background Sound for Voice Messaging System ", filed on July 31, 2003. 



2) We are also co-inventors of the invention described in claimed in U. S. Patent Application 
Number 09/782,773 (hereinafter, "the Parent Application"), now U. S. Patent 7,003,083 B2, from 
which the Subject Application claims priority. The Parent Application is also entitled "Selectable 
Audio and Mixed Background Sound for Voice Messaging System" and was filed on February 
13,2001. 
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3) We created an invention disclosure document (hereinafter, "Disclosure"), on which the 
above-identified application is based, in this country before October 6, 2000. A copy of this 
Disclosure is submitted herewith as "Exhibit A", and supporting information for the Disclosure is 
submitted herewith as "Exhibit B" and "Exhibit C". The Disclosure in Exhibit A was created on 
08/3 1/2000, as indicated in the rectangular box at the top of the first page (having heading 
"Disclosure RSW 8-2000-0177', which is the disclosure identifier of the Parent Application), and 
was submitted to our employer's Invention Development Team for review on 08/3 1/2000 (see 
"Submitted date" under "Summary" on the first page). The invention was workable on 
08/01/2000, as indicated in "Question 1" on Page 2 of the Disclosure. The supporting 
information in Exhibit B was archived on 09/12/2000, as indicated in the rectangular box with 
heading "Main Idea for Disclosure RSW 8-2000-0177'. Further details of this supporting 
information are provided in Exhibit C, which was also archived on 09/12/2000, as indicated in the 
rectangular box with heading "Main Idea for Disclosure RSW 8-2000-0149". Note that Exhibit 
B, as originally created, includes the supporting information in Exhibit C by reference thereto. 
See "Main Idea", item 1, of Exhibit B, which states "Reference disclosure RSW8-2000-0149" 
(i.e., the information presented herein as Exhibit C). See, in particular, the following portions of 
Exhibit C: "Disambiguating topics: Leaving a message" on pages 1 - 2; "Preparing the message 
for retrieval" on page 2; "Retrieving the message" on page 2; "Actions on partial voicemails" on 
page 2; "Audio only messages" on page 3, and "User selected audio background" on page 3. 

Portions of the Disclosure in Exhibit A and supporting information in Exhibits B and C, as 
well as Exhibits D - R discussed below, may be redacted to remove information not necessary to 
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establish the invention's conception or reduction to practice. 

Exhibit C indicates, at the bottom of Page 3, attachment of 4 files which were provided 
therewith. Copies of 2 of these files, "LvMsg.PRZ" and "msgctxt.prz", are submitted herewith as 
Exhibits D and E, respectively. 

Exhibits A - E establish that we conceived our invention, as recited in independent Claims 
1, 34, 100, 1 1 1, and 1 14 of the Subject Application, before October 6, 2000. 

4) The invention was diligently reduced to constructive practice in this country from prior to 
October 6, 2000 to the filing of the Parent Application on February 13, 2001. This diligence is 
evidenced by the following: 

a) The Invention Disclosure document and supporting information (including, inter 
alia, the information in Exhibits A - E) was mailed to a registered patent attorney on September 
13, 2000 (notably, prior to October 6, 2000), with a target filing date of January 31, 2001, for the 
purpose of preparing and filing a utility patent application, which led to the February 13, 2001 
filing of the Parent Application. See Exhibit F, providing a copy of the transmittal letter dated 
09/13/2000 to the registered patent attorney. 

b) Initial discussions of the invention of the Parent Application with the registered 
patent attorney occurred on December 8, 2000 and December 13- 14, 2000, and continuing into 
early January, 2001. See Exhibit G, which provides a copy of an email dated 12/08/2000 from 
Renee Kovales to the registered patent attorney. This email refers to "3 separate ideas", which 



Serial No. 10/632,177 



-3- 



RSW920000128US2 



became IBM Dockets RSW9-2000-0126, RSW9-2000-0127, and RSW9-2000-0128, the latter of 
which is the Parent Application, See also Exhibit H, which provides a copy of an email dated 
12/13/2000 from Renee Kovales to the registered patent attorney, and which provides additional 
discussion of these 3 IBM Dockets. Pages 4-5 of the email in Exhibit H pertain to the IBM 
Docket which is the Parent Application. Exhibit I provides a copy of an email from the registered 
patent attorney to Renee Kovales on 12/14/2000, discussing the Parent Application. Exhibits J, 
K, and L provide copies of an email dated 12/28/2000 and 2 emails dated 12/29/2000, 
respectively, from Renee Kovales to the registered patent attorney, providing additional 
discussion of the Parent Application (and each noting that charts were attached thereto). Exhibit 
M provides a copy of an email dated 01/03/2001 from the registered patent attorney, discussing a 
timeline for preparing the patent applications for the 3 IBM Dockets discussed with reference to 
Exhibits G and H (including the Parent Application). 

c) A first draft of the Parent Application was created by the registered patent attorney 
in early January, 2001 and distributed to us for review. Exhibit N provides a copy of an email 
dated 01/10/2001 from Edie Stern to the registered patent attorney, noting that comments ("First 
set of markups") are provided for "RSW9-2000-0128 first draft". 

d) A second draft of the Parent Application was created by the registered patent 
attorney and distributed to us for review on January 1 7, 200 1 . See Exhibit O, providing a copy of 
an email dated 01/17/2001 from the registered patent attorney, noting the subject as "RSW9- 
2000-0128 second draft". 

e) It should also be noted that we worked concurrently on the other 2 IBM Dockets 
which are referenced above with regard to Exhibits F and G during the above-discussed time 
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period. See Exhibit P, providing a copy of an email dated 01/22/2001 from the registered patent 
attorney to Edie Stern. (These IBM Dockets were also filed by IBM Corporation on February 
13, 2001, as U. S. Patent Applications 09/782,564 and 09/782,772.) 

f) A final draft of the Parent Application was sent to us by the registered patent 
attorney on January 31, 2001 . See Exhibit Q, providing a copy of an email dated 01/3 1/2001 
from the registered patent attorney, noting the subject as "RSW9-2000-0128 final draft". 

g) Following our review and approval, this final draft of the Parent Application was 
sent by the registered patent attorney to IBM Corporation on January 31, 2001 . See Exhibit R, 
providing a copy of a transmittal letter dated January 31, 2001 from the registered patent attorney 
to IBM Corporation. 

h) Following receipt of this final draft of the Parent Application by IBM Corporation, 
we executed an Oath/Declaration therefor on February 12, 2001; February 7, 2001; and February 
7, 2001, respectively, after which the Parent Application was filed by IBM Corporation on 
February 13, 2001. 

5) We hereby declare that all statements made herein of our own knowledge are true and that 
all statements made on information and belief are believed to be true; that these statements are 
made with the knowledge that willful false statements and the like so made are punishable by fine 
or imprisonment, or both, under 18 U. S. C. §1001; and that such willful statements may 
jeopardize the validity of the application or any patent issued thereon. 
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Edith H. Stem Date 



Barry E. Willner Date 
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Exhibit A 




Disclosure RSW8-2000-0177 

Prepared for and/or by an IBM Attorney - IBM Confidential 

Created By Renee Kovales On 08/31/2000 03:34:06 PM EDT 
Last Modified By wptsl wptsl On 01/07/2005 07:1 3:45 PM EST 
Archived on 12/16/2001 



Required fields are marked with the asterisk (*) and must be filled in to complete the form . 

*Trtle of disclosure (in English ) 

Selectable Background Sound 

Summary 



Status 



Final Decision (File) 



Final deadline 



Final deadline reason 



Docket family 



RSW9-2000-0128 



* Processing location 



Raleigh -RSW 



* Functional area 



Attorney/Patent professional 



(RSW) Horn: Technology Group (Kopkind,Lindquist) 



Jeanine Ray/Raleigh/IBM 



Invention development team (IDT) 



Marcia L Stockton/Raleigh/IBM 



Submitted date 



08/31/2000 04:17:56 PM EDT 



* Owning division 



Incentive program 



Lab 



* Technology code 



Patent value tool (PVT) score 



AIM 



Inventors with a Blue Pages entry 

Inventors: Renee Kovales/Raleigh/IBM, Barry Willner/Watson/IBM, Edie Stern/Fort Lauderdale/IBM 

Inventor Inventor 
Inventor Name Serial DFv/Dept Phone Manager Name 



Kovales, Renee 
Willner, Barry E. 
Stem, Edith H. 



10/M0FA N/A 
7H/M21A N/A 
22A/L7A N/A 



Pozefsky, D.P. (Diane) 
Huber, Judy B. 
Harrison, Colin G. 



> denotes primary contact 

Inventors without a Blue Pages entry 
Invention Development Team Information 



Attorney/Patent Jeanine Ray/Raleigh/IBM 

professional 

Invention development Marcia L Stockton/Raleigh/IBM@IBMUS 
team (IDT) 

Response due to IP&L 1 0/01 /2000 



Main Idea 



To view the Main Idea of this disclosure, open the "Main Idea" document from the view 
♦Critical Questions (Questions 1-9 must be answered in English) 
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'Question 1 

On what date was the invention workable? 08/01/2000 Please format the date as MM/DD/YYYY 
(Workable means Le. when you know that your design will solve the problem) 

'Question 2 ™ O Yes 

Is there any planned or actual publication or disclosure of your invention to • No 
anyone outside IBM? 

If yes, Enter the name of each publication or patent and the date published below. 

Publication/Patent: 
Date Published or Issued: 

Are you aware of any publications, products or patents that relate to this OYel 
invention? 9 No 

If yes, Enter the name of each publication or patent and the date published below. 

Publication/Patent: 
Date Published or Issued: 



'Question 3 


O Yes 


Has the subject matter of the invention or a product incorporating the invention 


• No 


been sold, used internally in manufacturing, announced for sale, or included in a 




proposal? 




Is a sale, use in manufacturing, product announcement, or proposal planned? 


O Yes 
• No 



If Yes, identify the product if known and indicate the date or planned date of sale, announcements, or 
proposal and to whom the sale, announcement or proposal has been or will be made. 

Product: 
Version/Release: 



Code Name: 
Date: 
To Whom: 

If more than one, use cut and paste and append as necessary in the field provided . 

•Question 4 O Yes 

Was the subject matter of your invention or a product incorporating your # no 

invention used in public, e.g., outside IBM or in the presence of non-IBMers? 

If yes, give a date. Please format the date as MM/DD/YYYY 



•Questions ^ — Ores 

Have you ever discussed your invention with others not employed at IBM? # No 

If yes, identify individuals and date discussed . Fill in the text area with the following information, the 
names of the individuals, the employer, date discussed, under CDA, and CDA #. 



•Question 6 Q Yes 

Was the invention, in any way, started or developed under a government • no 

contract or project? Q NotS ure 

If Yes, enter the contract number 



•Question 7 (J Yes 

Was the invention made in the course of any alliance, joint development or other • No 
contract activities? Q Not Sure 
If Yes, enter the following: 



Page 2 IBM Confidential Printed 10/19/2006 at 09:56:12 AM 



RSW8-2000-0177 Selectable Background Sound - continued 



Name of Alliance, Contractor or Joint Developer 
Contract ID number 

Relationship contact name — - 
Relationship contact E-mail 
Relationship contact phone 



'Question 8 ~ q Ye c 

Have you, or any of the other inventors, submitted this same invention • no 

disclosure or similar invention disclosure previously? 
If Yes, please provide disclosure number below: 



'Question 9 Q y es 

Are you, or any of the other inventors, aware of any related inventions O No 

disclosures submitted by anyone in IBM previously? 

If Yes, please provide the docket or disclosure number or any other identifying information below : 



Question 10 ^ — "~ - ■— _ — _ - 

What type of companies do you expect to compete with inventions of this type? Check all that apply. 

[...] Manufacturers of enterprise servers 
n Manufacturers of entry servers 
U Manufacturers of workstations 

Manufacturers of PC's 
t>3 Non-computer manufacturers 
L J Developers of operating systems 
□ Developers of networking software 
L J Developers of application software 
[ ...J Integrated solution providers 
[^3 Service providers 

Other (Please specify below) 



Question 11 ' ______ 

If the invention relates to a product or service that is outside the scope of your business unit, please 
recommend IBM business unit(s), IBM location(s) or individual(s) within IBM that you think would 
provide a good evaluation of your invention : 



♦Patent Value Tool (Optional - this may be used by the inventor and attorney to assist with tiie evaluati... 

(The Patent Value tool can be used by the inventor(s) to determine the potential licensing value of your 

invention.) 

Market 

'Question 1: What is the anticipated annual market size (in dollars) that will be captured by your 
Invention? 

Reason(s) for above Answer: 
Claims 
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•Question 1 : How new is the technical field? 
Reason(s) for above Answer: 

•Question 2: How central is the invention to the product(s) which might be expected to contain the 
invention? 

Reason(s) for above Answer: 

•Question 3: What is the scope of the claim? 

Reason(s) for above Answer: 

Portfolio Need 

•Question 1: What are the portfolio needs in the area of your invention? 
Reason(s) for above Answer: 
Exploitation & Enforcement 

•Question 1 : How easily can the use of the invention by a competitor be detected? 
Reason(s) for above Answer: 

•Question 2: How easily can the use of the invention be avoided by a competitor? 
Reason(s) for above Answer: 
Business Value 

•Question 1: What percentage of the companies producing products in the field of this invention might use 
this invention? 

Reason(s) for above Answer: 

•Question 2: What is the value of this patent to current or anticipated Alliance Activity between IBM and 
other companies? 

Reason(s) for above Answer: 

•Question 3: What is the value of this patent to current or anticipated Technology Transfer Activity 
between IBM and other companies? 

Reason(s) for above Answer: 

•Question 4: Does it result in prestige to IBM? 

Reason(s) for above Answer: 

Evaluation 

This team evaluation was entered by Marcia Peters/Raleigh/IBM on 09/01/2000 

What is the team's evaluation of this disclosure? Search 

Date evaluated : 08/31/2000 

Evaluation comments 

During the evaluation we broke RSW8-2000-0149 into 3 and rated them separately. (See 
RSW8-2000-01 76 and -01 77). 
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REDACTED 



This disclosure RSW8-2000-0177 pertains to mixing a selectable background sound into a voice message 
and it was also rated search but with a vote 4 in favor of search and 3 in favor of publish. 



Final Evaluation History 


Who made the final evaluation 


Final evaluation date 


Search 


Marcla Peters/Raleigh/IBM 


08/31/2000 



Search Information 



Date sent: Target completion date: Search results received date: 

Who was the search sent to (This area is to designate a Local Searcher name or WAIPL): 
^Search type: □ Patentability □ Clearance □ Validity □ State of A rt 

^Features to be searched: ~ ~" ~~ 



Search Office Information 



Target completion date: U Search has been delayed Ship/Return date: 

Search conducted by 

Comments — — 

Final Decision 

This decision was entered by Dianne Lane /Raleigh /IBM on 09/13/2000 

Decision: File Status: N/A 

PPM area: ~~ ~~ 



Date of final decision : 09/13/2000 



Additional filing information 

Planned Filing date: 01/31/2000 
Filing comments: Marcia Doubet 

Additional decision comments 
Final Decision History 



Entered on 13-Sep-2000 by Dianne Lane 

File N/A 13~Sep-2000 Docket Family: RSW920000128 



Post Disclosure Text & Drawings 
To add additional information related to this disclosure once it has been submitted, click the action button 
below and a new document will be opened for you to enter the new information . To view existing post 
disclosure information, double-click on the item in the list below (if there has been additional information 
entered), and the document will open for you to view. 
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Date entered Post disclosure comments and drawings (double-click an Item below to view) 



Form Revised (05/28/03) 
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Exhibit B 



Main Idea for Disclosure RSW8-2000-0177 - continued 



Main Idea for Disclosure RSW 8-2000-0177 

%frjf Prepared for and/or by an IBM Attorney - IBM Confidential 

§ Archived On 09/12/2000 01 :03:24 AM 



3* r, 



Title of disclosure (in English ) 

Selectable Background Sound 

Main Idea 

1. Describe your invention, stating the problem solved (if appropriate), and indicating the advantaqes of 
using the invention. 

Reference disclosure RSW8-2000-0149 

2. How does the invention solve the problem or achieve an advantage, (a description of "the invention" 
including figures inline as appropriate )? 



3. If the same advantage or problem has been identified by others (inside/outside IBM), how have those 
others solved it and does your solution differ and why is it better? 



4. If the invention is implemented in a product or prototype, include technical details, purpose, disclosure 
details to others and the date of that implementation. 
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Main Idea for Disclosure RSW8-2000-0149 - continued 




Main Idea for Disclosure RSW 8-2000-0149 

Prepared for and/or by an IBM Attorney - IBM Confidential 

Archived On 09/12/2000 01:02:10 AM 



Title of disclosure (in English) 

Audio Content Context and Enhancement 

Mam Idea 

1. Describe your invention, stating the problem solved (if appropriate), and indicating the advantages of 
using the invention . 

Our invention, audio content context and enhancement, is a system and method to provide additional 
context during audio messages, and to provide disambiguation of topics during audio messages . Further 
it allows "skimming" of voice mail analogous to a reader skimming the contents of an e-mail message. 

Face to face communication between people involves many parallel communication paths . We derive 
information from body language, from words, from intonation, from facial expressions, from the distance 
between our bodies. Distance communication generally involves only a few of these paths, with users 
trying to overcome the limitations so imposed. Today, unified messaging and network convergence 
exacerbates the problem by adding the difficulties of media transformation to our communication . 

E-mail tries to overcome these problems by using font sizes, colors and emoticons (combination of text 
symbols which bear a semblance to facial expressions ) to express non-text information. This information 
includes emphasis, emotion, irony, etc. When e-mail is transformed via an audio read out, as is done in 
unified messaging, this context can be lost. In email, the paragraph structure and use of holding and 
italics gives clues as to the number and importance of topics . Thus, e-mail tries to overcome the 
limitations of distance by using visual clues for both semantic and contextual meaning . Further, visual 
cues such as paragraph distinctions pertaining to the end of the message on the page are perceivable as 
soon as the page is displayed, providing a "broadside" perception of the message. 

Voicemail has a different set of problems. When equivalent information is left via voicemafl, while the 
recipient has the benefit of the nuances available through voice, the recipient does not have the 
advantages of the other parallel forms of communication available in person . Unlike e-mail, with 
voicemail the recipient does not have the advantage of broadside perception of the message . Our 
invention, audio content context and enhancement, helps to overcome both these limitations . 

2. How does the invention solve the problem or achieve an advantage, (a description of "the invention", 
including figures inline as appropriate)? 

Our invention is a method and system for disambiguating topics in an audio message and for 
communicating context associated with a message, or part of a message, with an audio cue (eg level of 
certainty of translation may be represented by a meaningful background tone). The invention enables 
rapid navigation of voicemail messages from an ordinary, non -screen, telephone, and rapid retrieval of 
phone numbers from the middle of a voicemail message. This invention also enables messages to be left 
with "favorite music" tags. 

Base level is delineation (disambiguate) 
Next level is selection of context 
Personal choice of audio (theme song) 

Disambiguating topics: 

Leaving a message - Speakers tend to be less precise in oral speech than they are in written 
communications. This can be inefficient when communicating via voicemail . Our invention includes a 
method and system for such speakers to signal, with a keyed alert sequence (KAS) during the course of 
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Main Idea for Disclosure RSW8-2G00-0149 - continued 



the message they are leaving, that they are now changing topic, or marking a particularly important 
passage. This is accomplished with a sequence of keystrokes on the telephone keypad . For example, in 
a message covering two topics, the speaker may pause, key in the KAS (e.g., *), and continue speaking. 
The KAS alerts the voicemail system that a new topic has been started. If a very important passage is 
about to be communicated, e.g., a phone number or identifier, the speaker may key in the appropriate 
alert (e.g., * or perhaps another sequence such as *9) before communicating the information. 

Note: In the above description, the KAS may be recorded as part of the audio stream. In an alternate 
embodiment, KAS detection causes the development of metadata associated with the audio stream which 
allows the voicemail system to index to the KAS points . 

Preparing the message for retrieval - The voicemail system analyzes each message for KAS tones . If 
such tones have been used, the voicemail system can associate an audio cue with each section of the 
message so delineated. The associated cues are mixed in with the message, i.e., "conferenced". 
Alternatively, the voicemail system may alternate between two such audio cues, with adjacent sections 
each having one of the cues. Such cues may be musical, for example tunes . They may be natural 
sounds such as birdsong, ocean waves. One cue may be high pitched and the next low pitched, etc. The 
information being conveyed is in the parsing of the topics. The voicemail system can then index the 
message so that the KAS tones form a series of navigable points within the message. In another 
embodiment, the association of audio cues with the sections of the message can be done as the message 
is retrieved, by either the voicemail system or the client telephone equipment (e.g., the cellphone). 
Retrieving the message -When a user acts to receive the message, the voicemail system can announce 
or otherwise convey (e.g., with a numerical indicator on a cell phone display) the existence, and the 
number of KAS delineated sections. This gives the user a sense of topics to be covered . That is, along 
with the date and time, and number of origin of the message, the number of topics can also be announced . 
Alternatively, the audio cues can indicate to the user the continuity of the topic . When the cues change, 
the topic has changed. 

Forward to the next topic - This invention enables a new voicemail feature : forward to the next topic. 
Each topic is delineated with a KAS and is navigable. Thus, the user can listen to the voicemail serially, 
or can listen to a few seconds of each topic, and skip to the next as necessary. This is particularly useful 
during replay of messages. When a user has listened to a message, and potentially missed noting a 
phone number for example, then if that phone number had been noted with a KAS, the user can navigate 
to it more easily. 

Actions on partial vofcema/ts -With each section delineated, this invention enables new voicemail feature 
which act on each section individually . Any action that can be taken on a voicemail can be taken on a 
partial voicemail. The user can delete or forward partial voicemails. At the end or beginning of a KAS 
delineated topic, the user can elect to forward only the previous section or the upcoming section to a third 
party. This usefully allows the pertinent portion of a message to be forwarded, while allowing the user to 
retain privacy on the remainder of the message. Similarly, the user can choose to delete one or more 
delineated sections of a voicemail, retaining only those portions which he or she desires to save . 

Media Transformation: An audio cue can be used to minimize the effect of a media transformation from a 
non-audio source such as text. In one embodiment, each paragraph separation is taken to be a KAS as 
above, and an audio cue is mixed into the message. In another embodiment, an appropriate audio file is 
mixed with the resultant audio from the transformation, and thereby providing parallel information as to 
context. In this invention, we are not inserting an audio sound or cue in-line in the message (e.g., a giggle 
sound in place of a smiley emoticon), but we are "conferencing" an appropriate audio file for part of, or all 
of, the message. An appropriate cue may be determined in several ways. If the originating source has 
supplied keywords, this can be used as a source of cueing . If the originating source has not supplied 
keywords, (e.g., today's e-mail systems), the transformation may select an appropriate noun in the first 
few sentences to use as a source of cueing. For example, if the first sentence of a paragraph reads The 
wedding date has been set.", the appropriate audio cue may be church bells. If the sentence reads "The 
meeting was very productive" the appropriate audio cue may be papers rustling, and low conversation . 
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Other Information: Audio cues can provide additional contextual information, especially for 
transformations. For example, when machine language translation is involved, audio cues can be used to 
indicate degree of certainty in the translation . A background hum might indicate certainty of translation 
with higher pitches indicating more certainty and lower pitches indicating less . Note that this may be 
usefully employed in an audio to audio transform, with voice recognition in the middle, or a text to speech 
plus language transform. Another example would be in the use of speaker identification to identfly the 
speaker leaving a voicemail. The background tone can indicate the degree of certainty in the 
identification. Similar methods can be used to highlight passages in text as to degree of certainty, when 
for instance the text was obtained through voice recognition . Another example, in a text to speech 
system an audio cue could be used to indicate the color of the text (which may indicate the degree of 
importance, a change in topic, and etc.); the background hum as described above could be applied in this 
case with the pitch indicating the level of importance, a different topic, or more simply a change in color . 

Audio only messages: \x\ another embodiment, we associate an audio emotional cue with an audio 
message. In this embodiment, the audio file is selected by the speaker to be mixed with the message, or 
inserted in the message directly without transformation . The file selected represents the context which the 
user wishes to convey. For example, for an angry message, the user may select quiet rolling thunder as a 
background. The user can select this effect before the message, during the message, or after the 
message is recorded with appropriate DTMF tones on the telephone keypad . This can also be done for 
parts of the message, as delineated with KAS tones. 

User selected audio background :\x\ this embodiment, the speaker selects the audio background as part of 
the message being left. For example, the speaker may select from a number of offered backgrounds 
("top 40") what music is to play in the background of the message. The music is mixed with the message, 
as the other audio cues above. In an alternate embodiment, the speaker's telephone client receives the 
selection request and communicates it to the voice mail system . If the speaker has unique or favorite 
music (for example as is used to provide ringing tone on a cell phone), and wishes to have this familiar 
identification (e.g., a speaker's "theme song") as part of the message being left, the telephone can 
communicate the music, or identify the music to the receiving voicemail system . 

Note that this embodiment enables a business model of merchandizing play out of popular music, via a 
subscription, or a pay per use of the music. In this model, it is expected that the speaker would pay for the 
music, as the receiver has no opportunity to accept or decline. This is consistent with the Calling Party 
Pays model standard in telephony. Again, the music may be transmitted from the phone, identified from 
the phone, or supplied by the voicemail system from a control indicator sent by the phone . Note that 
Sonera has a lucrative business of downloading ringing music to phones for 50 cents a crack; this is the 
equivalent for leaving messages. 

Fiaurel: Call flow leaving a message with delineation (including finding the phone number) 

H 

LvMsg.PRZ 

Fi gure 2 : Playing a voicemail which has delineation 



playvm.prz 

Fi gure 3 : Leaving a message with context 



msgctxtprz 

Figure^Histogram for the GUI; GUI is web driven for example 

US 

AudioSegmentsPRI 
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Main Idea for Disclosure RSW8-2000-0149 - continued 



The invention outlined in this document has the following advantages /differences' 

1 . Allows caller to associate background sounds with the message . 

2. Is designed to improve a voice mail system and therefore has options that are applicable in a voice 
mail system only (such as forwarding message segments to another user, storing only some 
segments while deleting others), y 

3. Allows for indications of translation certainty via background sound 



REDACTED 



Claims 



Claim 1: a system and method for providing message sectioning comprising 

reception of speech 

delineation of sections of speech 

association of sections of speech with audio cue 
Claim 1 A: the system of claim 1 where 

KAS tones are captured 

sections are delineated by KAS tones 
Claim 2; where association is done by voicemail system before retrieval 
Claim 3: where audio cue is chosen by the speaker 
Claim 4: where audio cue is chosen by the voicemail system 



Claim 5: a system and method for providing message context comprising 
user message 
determination of context 
association of context with an audio file 

creation of a new message composed of user message and audio file 
new message transmission 

Claim 5A: the method of claim 5 where 

determination of context is performed by the user 

ClaimBB: the method of claim 5 where 

determination of context is based on degree of certainty of the message data 

Claim 5C the method of claim 5B where 

the message data is the content of the message 

Claim 5D the method of claim 5 where 

the message data is the identification of the originator of the message 
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Claim 6: The method of claim 5 where 

the user message is text 
Claim 7: The method of claim 5 where 

the message context is an emoticon 
Claim 8: The method of claim 6 where 

the new message is a transformation to audio 
Claim 9: The method of claim 8 where 

the audio file is mixed with the verbalization of the text for the duration of the verbalization 
Claim 6: The method of claim 8 where 

the audio file is longer than the duration of the verbalization 
Claim n: The method of claim 5 where the user message is verbal (such as voice mail) 

The method of claim n where the selection of the context is made after the recording 
The method of claim n where the audio file is mixed with the message for the duration of the 
message 

The method of claim n where the audio file is longer 

Claim 10: The system of claim 1 where 

sections are individually responsive to voicemaii commands 
Claim 1 1 : The system of claim 1 0 where 

sections respond to forwarding 
Claim 12: The system of claim 10 where 

sections respond to deleting 

Claim 13 a system and method for providing message sectioning comprising 
reception of speech 
reception of audio cues 
delineation of sections of speech 

Claim 14 the method of claim 13 further comprising 

association of sections of speech with audio cues 

Claim 15: The method of claim 1 where 

delineation of sections of audio is done in line 

Claim 16: The method of claim 1 where 

delineation of sections of audio is done via associated metadata 



(siomilar dependent claims) 



3. If the same advantage or problem has been identified by others (Inside/outside IBM), how have those 
others solved it and does your solution differ and why is it better? 

Today, users of voicemaii from a telephone do not have the ability to navigate within the voicemaii in a 
controlled fashion. Some systems allow the user to speed up the message, or skip the message, but none 
allow the originator of the message to specify where the topics lie . This enables semantic navigation . 

4. If the invention is implemented in a product or prototype, include technical details, purpose, disclosure 
details to others and the date of that implementation. 

na 
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Exhibit F 



Li. ; Dianne Lane 

I- <W "" 09/13/2000 01:55 PM 

To: Marcia Doubet/Raleigh/Contr/IBM@ibmus 

cc: 

From: Dianne Lane/Raieigh/IBM@IBMUS 

Subject: Dockets RSW9-2000-0126, 0127 and 0128 



Hey, Marcia. Per your conversation with Jeanine I've set up three new dockets and am forwarding the 
disclosure materials to you today by first class mail. Renee Kovales will act as lead inventor for all three 
applications. Her number is 254-4522. These applications need to be filed by January 31, 2001 Thanks 
much! 



Dianne Lane 

IBM Software Group Intellectual Property Law 
Phone: (919) 543-6383 (TL441) 
Fax: (919)254-4330 (TL444) 

Visit the RTP Legal Web Site at: http://w3legal.raleigh.ibm.com/index.html 
for answers to common legal questions. 
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Renee Kovales, 11:38 AM 10/12/20, Fw: Patent info 



To: w Marcia L. Doubet" <mltitoinaspring . com> 
Subject: Fw: Patent info 

X-Mailer: Lotus Notes Release 7.0 HF242 April 21, 2006 
From: Renee Kovales <rkovales@us.ibm.com> 
Date: Fri, 12 Oct 2007 11:38:56 -0400 

X-MIMETrack: Serialize by Router on D03NM118/03/M/IBM (Release 8.0 (August 02, 2007) at 
10/12/2007 
09:3B:b8, 

Serialize complete at 10/12/2007 09:38:58 
X-ELNK-Received-Info: spv=0; 
X-ELNK-AV: 0 

X-ELNK-Info: sbv=0; sbrc=.0; sbf=00; sbw-000; 



Renee Kovales 

Research Triangle Park, NC 27709 
(919) 254-4522, t/1 444-4522 

— Forwarded by Renee Kovales/Raleigh/IBM on 10/12/2007 1 1:40 AM — 
Renee Kovales/Raleigh/IBM 

12/08/2000 10:12 AM 

To 

Marcia Doubet/Rateigh/Contr/IBM@ibmus 
Patent info 



cc 

Subject 



Hi Marcia, 

Originally we started with 1 idea and then it was split into 3 separate ideas when we brought our idea to the review board. I haven't updated 
the original text to split the information apart, but can do so. I've included the summary of each idea and the original text, below. Let me 
know if this is what you need and how I can help. I have a lot of vacation in December, but can plan to work on this as required. Just let me 
know what your schedule is for working on this in December. Also, I will be at a conference 1/14/01-1/21/01 . 

Here is a summary of the breakout: 

RSW8-2000-0149 Audio Context Content and Enhancement (Renee Kovales et al. x44522) Split into 3 disclosures, all 3 rated search (please 
submit 2 additional WPTS disclosures). 

* Recording voice mail and leaving freeform bookmarks ; during playback, method of 
navigating through speech containing freeform bookmarks . 

* Text to speech - audio renderings for expressing text nuances (bold, font, color, all 
caps, emoticons, etc.) in speech. Additionally, the use of stylesheets to configure the 
rendering process . If the claims include this stylesheets idea, Jim Matthewson should 
be named as a co-inventor. IP Law may choose to split this into multiple patents. 

* Selectable mixed background sound for voice messaging system. 

* NOTE: We'd like this team of inventors to bring in another disclosure on the use of 
hypertext links in speech in a voice recognition or text-to-speech system, e.g. phone 
numbers (do you want me to dial this number?), URLs (do you want me to read you this 
website) . . . etc. 



Here is the text that we submitted: 

1. Describe your invention, stating the problem solved (if appropriate), and indicating the 
advantages of using the invention. 

Our invention, audio content context and enhancement, is a system and method to provide additional context during audio messages, and to 
provide disambiguation of topics during audio messages. Further it allows "skimming" of voice mail analogous to a reader skimming the 
contents of an e-mail message. 

Face to face communication between people involves many parallel communication paths. We derive information from body language, from 
words, from intonation, from facial expressions, from the distance between our bodies. Distance communication generally involves only a few 
of these paths, with users trying to overcome the limitations so imposed. Today, unified messaging and network convergence exacerbates 
the problem by adding the difficulties of media transformation to our communication. 
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E-mail tries to overcome these problems by using font sizes, colors and emoticons (combination of text symbols which bear a semblance to 
taaai expressions) to express non-text information. This information includes emphasis, emotion, irony, etc. When e-mail is transformed via 
an audio read out as is done in unified messaging, this context can be tost. In email, the paragraph structure and use of holding and italics 
gives dues as to the number and importance of topics. Thus, e-mail tries to overcome the limitations of distance by using visual clues for 
both semantic and contextual meaning. Further, visual cues such as paragraph distinctions pertaining to the end of the message on the page 
are perceivable as soon as the page is displayed, providing a "broadside" perception of the message. 

Voicemail has a different set of problems. When equivalent information is left via voicemaH, while the recipient has the benefit of the nuances 
available through voice, the recipient does not have the advantages of the other parallel forms of communication available in person Unlike 
e-mail, with voicemail the recipient does not have the advantage of broadside perception of the message. Our invention audio content 
context and enhancement, helps to overcome both these limitations. 

2. How does the invention solve the problem or achieve an advantage,(a description of 'the invention", including figures inline as 
appropriate)? 

Our invention is a method and system for disambiguating topics in an audio message and for communicating context associated with a 
message, or part of a message, with an audio cue (eg level of certainty of translation may be represented by a meaningful background tone). 
The invention enables rapid navigation of voicemail messages from an ordinary, non-screen, telephone, and rapid retrieval of phone numbers 
from the middle of a voicemail message. This invention also enables messages to be left with "favorite music" tags. 

Base level is delineation (disambiguate) 
Next level is selection of context 
Personal choice of audio (theme song) 

Disambiguating topics: 

Leaving a message - Speakers tend to be less precise in oral speech than they are in written 
communications. This can be inefficient when communicating via voicemail. Our invention 
includes a method and system for such speakers to signal, with a keyed alert sequence (KAS) 
during the course of the message they are leaving, that they are now changing topic, or 
marking a particularly important passage. This is accomplished with a sequence of keystrokes 
on the telephone keypad. For example, in a message covering two topics, the speaker may 
pause, key in the KAS (e.g., *), and continue speaking. The KAS alerts the voicemail system 
that a new topic has been started. If a very important passage is about to be communicated, 
e.g., a phone number or identifier, the speaker may key in the appropriate alert (e.g., *or 
perhaps another sequence such as *9) before communicating the information. 

Note: In the above description, the KAS may be recorded as part of the audio stream. In an alternate embodiment, KAS detection causes 
the development of metadata associated with the audio stream which allows the voicemaH system to index to the KAS points. 

Preparing the message tor retrieval - The voicemail system analyzes each message for KAS 
tones. If such tones have been used, the voicemail system can associate an audio cue with 
each section of the message so delineated. The associated cues are mixed in with the 
message, i.e., "conf erenced" . Alternatively, the voicemail system may alternate between two 
such audio cues, with adjacent sections each having one of the cues. Such cues may be 
musical, for example tunes. They may be natural sounds such as birdsong, ocean waves. One 
cue may be high pitched and the next low pitched, etc. The information being conveyed is in 
the parsing of the topics. The voicemail system can then index the message so that the KAS 
tones form a series of navigable points within the message. In another embodiment, the 
association of audio cues with the sections of the message can be done as the message is 
retrieved, by either the voicemail system or the client telephone equipment { e . g ., the cellphone). 
Retrieving the message - When a user acts to receive the message, the voicemail system can 
announce or otherwise convey (e.g., with a numerical indicator on a cell phone display) the existence, and the number of KAS 
delineated sections. This gives the user a sense of topics to be covered. That is, along with the date and time, and number of origin of the 
message, the number of topics can also be announced. Alternatively, the audio cues can indicate to the user the continuity of the topic. 
When the cues change, the topic has changed. 

Forward to the next topic - This invention enables a new voicema.il feature: forward to the 
next topic. Each topic is delineated with a KAS and is navigable. Thus, the user can 
listen to the voicemail serially, or can listen to a few seconds of each topic, and skip to 
the next as necessary. This is particularly useful during replay of messages . When a user 
has listened to a message, and potentially missed noting a phone number for example, then if 
that phone number had been noted with a KAS, the user can navigate to it more easily. 

Actions on partial voicemails ~ With each section delineated, this invention enables new 
voicemail feature which act on each section individually. Any action that can be taken on 
a voicemail can be taken on a partial voicemail . The user can delete or forward partial 
voicemails. At the end or beginning of a KAS delineated topic, the user can elect to 
forward only the previous section or the upcoming section to a third party. This usefully 
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allows the pertinent portion of a message to be forwarded, while allowing the user to retain 
privacy on the remainder of the message. Similarly, the user can choose to delete one or 
more delineated sections of a voicemail, retaining only those portions which he or she 
desires to save. 

Media Transformation: An audio cue can be used to minimize the effect of a media 
transrormatron from a non-audio source such as text. In one embodiment, each paragraph 
separation is taken to be a KAS as above, and an audio cue is mixed into the message In 
another embodiment, an appropriate audio file is mixed with the resultant audio from the 
transformation, and thereby providing parallel information as to context. In this invention, 
we are not inserting an audio sound or cue in-line in the message (e.g., a giggle sound in 
place of a smiley emoticon) , but we are "conferencing" an appropriate audio file for part of, 
or ail ot, the message. An appropriate cue may be determined in several ways. If the 
originating source has supplied keywords, this can be used as a source of cueing. If the 
originating source has not supplied keywords, (e.g., today's e-mail systems), the 
transformation may select an appropriate noun in the first few sentences to use as a source 
of cueing. For example, if the first sentence of a paragraph reads "The wedding date has 
been set.", the appropriate audio cue may be church bells. If the sentence reads "The 
meeting was very productive" the appropriate audio cue may be papers rustling, and low 
conversation. 

Other Information: Audio cues can provide additional contextual information, especially for 
transformations. For example, when machine language translation is involved, audio cues can 
be used to indicate degree of certainty in the translation. A background hum might indicate 
certainty of translation, with higher pitches indicating more certainty and lower pitches 
indicating less. Note that this may be usefully employed in an audio to audio transform, with 
voice recognition in the middle, or a text to speech plus language transform. Another 
example would be in the use of speaker identification to identfiy the speaker leaving a 
voicemail. The background tone can indicate the degree of certainty in the identification. 
Similar methods can be used to highlight passages in text as to degree of certainty, when for 
instance the text was obtained through voice recognition. Another example, in a text to 
speech system an audio cue could be used to indicate the color of the text (which may 
indicate the degree of importance, a change in topic, and etc.); the background hum as 
described above could be applied in this case with the pitch indicating the level of 
importance, a different topic, or more simply a change in color. 

Audio only messages: In another embodiment, we associate an audio emotional cue with an 
audio message. In this embodiment, the audio file is selected by the speaker to be mixed 
with the message, or inserted in the message directly without transformation. The file 
selected represents the context which the user wishes to convey. For example, for an angry 
message, the user may select quiet rolling thunder as a background. The user can select this 
effect before the message, during the message, or after the message is recorded with 
appropriate DTMF tones on the telephone keypad. This can also be done tor parts ot the 
message, as delineated with KAS tones. 

User selected audio background: In this embodiment, the speaker selects the audio background 
as part of the message being left. For example, the speaker may select from a number of 
offered backgrounds ("top 40") what music is to play in the background of the message. The 
music is mixed with the message, as the other audio cues above. In an alternate embodiment, 
the speakers telephone client receives the selection request and communicates it to the 
voice mail system. If the speaker has unique or favorite music (for example as is used to 
provide ringing tone on a cell phone), and wishes to have this familiar identification (e.g., 
a speaker's "theme song") as part of the message being left, the telephone can communicate 
the music, or identify the music to the receiving voicemail system. 

Note that this embodiment enables a business model of merchandizing play out of popular music, via a subscription, or a pay per use of the 
music. In this model, it is expected that the speaker would pay for the music, as the receiver has no opportunity to accept or decline. This is 
consistent with the Calling Party Pays model standard in telephony. Again, the music may be transmitted from the phone, identified from the 
phone, or supplied by the voicemail system from a control indicator sent by the phone. £ QyQ^Q"y £1 Q 

Figure 1 : Call flow leaving a message with delineation (including finding the phone number). Detached LvMsgPRZ 
Figure 2: Playing a voicemail which has delineation. Detached playvm.prz 
Figure 3: Leaving a message with context. Detached msgctxt.prz 
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Figure 4: Histogram for the GUI; GUI is web driven for example. Detached AudioSegments.prz 



Prior Art 

Emoticons - these are combinations of text used in e-mail messages to suggest an emotional 
context ror the text. For example: :) suggests a smile, sideways. :( suggests a sad face or 
frown. Emoticon information can be found on the web at websites such as: 

www.squareonetech.com/emoticon.html and www.bmiglobal.com/emoticon.htm. Additionally, books 
have been written on the subject; for example, Smileys by David W. Sanderson. 

.WAV files - .WAV files can be used in text/ graphics messages to add an emotional element 
to tne text. ror example, .wav files can include laughing, clapping, and etc. In current 
implementations, these .wav files are positional. AOL's chat sessions have the capability 
to play a .wav file for others on the chat (assuming they have sound capabilities) . One 
such chat session is The Herd ( h t tp : //members .aol. com/ earl werks/herd . h tm ) - which even posts 
a set of .wav files for chat members. Internet Relay Chat (IRC) is another example of 
chats with sound capabilities . 

In text-to-speech systems, emoticons, when encountered, can be either read a character at a time, are interpreted into a description (e.g., 
"Smiley Face" is announced for :) ), or play the appropriate sound accompanying the emoticon (e.g., giggle is heard when the giggle emoticon 
is encountered). 

NotesBuddy found on www.alphaworks.ibm.com interprets emoticons as the actual characters. 

In IBM's ViaVoice, users can define how the system will say an emoticon - this would be added to the user's dictionary. 

An e-mail discussing the idea of generating the emoticon to the equivalent sound can be found at http://iists.w3.org/Archives/Pubtic/w3c-wai- 
iq/1998JulSeo/0349.html 

http://www.beiMabs.com/proiect/tts/voices.html has TTS synthesis which reads emoticons as the actual characters (a demo of their product 
is on this web page). 



REDACTED 



The invention outlined in this document has the following advantages/differences: 

* Allows caller to associate background sounds with the message. 

* Is designed to improve a voice mail system and therefore has options that are applicable in a voice mail system only (such as 
forwarding message segments to another user, storing only some segments while deleting others). 

* Allows for indications of translation certainty via background sound 
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REDACTED 



Claims 



Claim 1: a system and method for providing message sectioning comprising 

reception of speech 

delineation of sections of speech 

association of sections of speech with audio cue 
Claim 1 A: the system of claim 1 where 

KAS tones are captured 

sections are delineated by KAS tones 
Claim 2: where association is done by voicemail system before retrieval 
Claim 3: where audio cue is chosen by the speaker 
Claim 4: where audio cue is chosen by the voicemail system 

Claim 5: a system and method for providing message context comprising 
user message 
determination of context 
association of context with an audio file 

creation of a new message composed of user message and audio file 
new message transmission 

Claim 5A: the method of claim 5 where 

determination of context is performed by the user 

CiaimSB: the method of claim 5 where 

determination of context is based on degree of certainty of the message data 

Claim 5C the method of claim 5B where 

the message data is the content of the message 

Claim 5D the method of claim 5 where 

the message data is the identification of the originator of the message 

Claim 6: The method of claim 5 where 

the user message is text 
Claim 7: The method of claim 5 where 

the message context is an emoticon 
Claim 8: The method of claim 6 where 

the new message is a transformation to audio 
Claim 9: The method of claim 8 where 

the audio file is mixed with the verbalization of the text for the duration of the verbalization 
Claim 6: The method of claim 8 where 

the audio file is longer than the duration of the verbalization 
Claim n: The method of claim 5 where the user message is verbal (such as voice mail) 
The method of claim n where the selection of the context is made after the recording 
The method of claim n where the audio file is mixed with the message for the duration of the message 
The method of claim n where the audio file is longer 

Claim 10: The system of claim 1 where 

sections are individually responsive to voicemail commands 
Claim 1 1 : The system of claim 1 0 where 

sections respond to forwarding 
Claim 12: The system of claim 10 where 

sections respond to deleting 

Claim 1 3 a system and method for providing message sectioning comprising 
reception of speech 
reception of audio cues 
delineation of sections of speech 
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Claim 14 the method of claim 1 3 further comprising 

association of sections of speech with audio cues 

Claim 15: The method of claim 1 where 

delineation of sections of audio is done m line 

Claim 16: The method of claim 1 where 

delineation of sections of audio is done via associated metadata 



(siomilar dependent claims) 



3. If the same advantage or problem has been identified by others (inside/outside IBM), how have those others solved it and does your 
solution differ and why is it better? 

Today, users of voicemail from a telephone do not have the ability to navigate within the voicemafl in a controlled fashion. Some systems 
allow the user to speed up the message, or skip the message, but none allow the originator of the message to specify where the topics lie 
This enables semantic navigation. 

4. If the invention is implemented in a product or prototype, include technical details, purpose, disclosure details to others and the date of that 
implementation. 

na 



Renee 

Renee Kovales 

Research Triangle Park, NC 
(919) 254-4522, t/1 444-4522 
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To: "Marcia L. Doubet*' <mldgmindspring . com> 
Subject: Fw: Patent info 

X-Mailer: Lotus Notes Release 7.0 HF242 April 21, 2006 
From: Renee Kovales <rkovales£us . ibm. com> 
Date: Fri, 12 Oct 2007 uy:39:38 -0600 

10/12/2007 k: SerialiZe by Router on D03NM118/03/M/IBM (Release 8.0 | August 02, 2007) at 

09:39:41 
X-ELNK-AV: 0 

X-ELNK-Inf o : sbv=0; sbrc=.0; sbf-00; sbw=000; 



Renee Kovales 

Research Triangle Park, NC 27709 
(919) 254-4522, t/1 444-4522 

— Forwarded by Renee Kovales/Raleigh/IBM on 10/12/2007 1 1:40 AM — 
Renee Kovales /Raleigh/ IBM 

12/13/2000 04:13 PM 

To 



Marcia Doubet/Raleigh/Contr/IBM 
Patent info 

Hi Marcia, 

I took a pass on separating this out. but I think I need to work on it some more. _ _ — a ^^^^ 

REDACTED 

Renee 



cc 

Subject 



Recording and Receiving Voice Mail with Preeform Bookmarks: 

Description of the Problem: 

Face to face communication between people involves many parallel communication paths. We derive information from body language, from 
words, from intonation, from facial expressions, from the distance between our bodies. Distance communication generally involves only a few 
of these paths, with users trying to overcome the limitations so imposed. 

E-mail tries to overcome these problems by using font sizes, colors and emoticons (combination of text syrnbote which bear a semblance to 
facial expressions) to express non-text information. This information includes emphasis, emotion, irony, etc. In e-mail, the paragraph 
structure and use of bolding and italics gives clues as to the number and importance of topics. Thus, e-mail tries to overcome the limitations 
of distance by using visual clues for both semantic and contextual meaning. Further, visual cues such as paragraph distinctions are 
perceivable as soon as the page is displayed, providing a "broadside" perception of the message. 

Voicemail has a different set of problems. While the recipient has the benefit of the nuances available through voice, the recipient does not 
have the advantages of the other parallel forms of communication available in person. Unlike e-mail, with voicemail the recipient does not 
have the advantage of broadside perception of the message. Additionally, the recipient is hampered in the retrieval of and subsequent 
actions on the message due to the inability to act on parts of the voice mail message, as can be done with e-mail. 

Our invention, Recording and Receiving Voice Mail with Freeform Bookmarks, helps to overcome these limitations, and defines a more 
flexible voice mail system. 

Description of the Invention: 

Our invention, Recording and Receiving Voice Mail with Freeform Bookmarks, is a system and method to provide disambiguation of topics 
during audio messages. Further it allows "skimming" of voice mail analogous to a reader skimming the contents of an e-mail message. This 
invention provides for a more flexible voice mail system. The invention enables rapid navigation of voicemail messages from an ordinary, 
non-screen, telephone, and rapid retrieval of phone numbers or special text from the middle of a voice mail message. Overall, this invention 
provides for a more flexible voice mail system. 

Disambiguating topics: 

Leaving a message - Speakers tend to be less precise in oral speech than they are in written 
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communications. This can be inefficient when communicating via voicemail. Our invention 
includes a method and system for such speakers to signal, with a keyed alert sequence (KAS) 
during the course of the message they are leaving, that they are now changing topic, or 
marking a particularly important passage. This is accomplished with a sequence of keystrokes 
on the telephone keypad. For example, in a message covering two topics, the speaker may 
pause, key in the KAS (e.g., *) , and continue speaking. The KAS alerts the voicemail system 
that a new topic has been started. If a very important passage is about to be communicated, 
e.g., a phone number or identifier, the speaker may key in the appropriate alert (e.g., * or 
perhaps another sequence such as *9) before communicating the information. 

Note: In the above description, the KAS may be recorded as part of the audio stream. In an alternate embodiment, KAS detection causes 
the development of metadata associated with the audio stream which allows the voicemail system to index to the KAS points. 

Figure 1: Call flow leaving a message with delineation (including finding the phone number) 

Preparing the message for retrieval - The voicemail system analyzes each message for KAS 
tones. If such tones have been used, the voicemail system can associate an audio cue with 
each section of the message so delineated. The associated cues are mixed in with the 
message, i.e., "confer ©need" . Alternatively, the voicemail system may alternate between two 
such audio cues, with adjacent sections each having one of the cues. Such cues may be 
musical, for example tunes. They may be natural sounds such as birdsong, ocean waves. One 
cue may be high pitched and the next low pitched, etc. The information being conveyed is in 
the parsing of the topics. The voicemail system can then index the message so that the KAS 
tones form a series of navigable points within the message. In another embodiment, the 
association of audio cues with the sections of the message can be done as the message is 
retrieved, by either the voicemail system or the client telephone equipment (e.g., the 
cellphone) . 

Retrieving the message - When a user acts to receive the message, the voicemail system can 
announce or otherwise convey (e.g., with a numerical indicator on a cell phone display) the 
existence, and the number of KAS delineated sections. This gives the user a sense of the 
topics to be covered. That is, along with the date and time, and number of origin of the 
message, the number of topics can also be announced. Alternatively, the audio cues can 
indicate to the user the continuity of the topic. When the cues change, the topic has 
changed. 

Forward to the next topic - This invention enables a new voice mail feature: forward to the 
next topic or each marked section. Each topic is delineated with a KAS and is navigable. 
Thus, the user can listen to the voicemail serially, or can listen to a few seconds of each 
topic, and skip to the next as necessary. This is particularly useful during replay of 
messages. When a user has listened to a message, and potentially missed noting a phone 
number for example, then if that phone number had been noted with a KAS, the user can 
navigate to it more easily. 

Actions on partial voicemails -With each section delineated, this invention enables new 
voice mail feature which act on each section individually. Any action that can be taken on 
a voice mail message can be taken on a section of voice mail. The user can delete or forward 
partial voice mails. At the end or beginning of a KAS delineated topic, the user can elect 
to forward only the previous section or the upcoming section to a third party. This 
usefully allows the pertinent portion of a message to be forwarded, while allowing the user 
to retain privacy on the remainder of the message. Similarly, the user can choose to delete 
one or more delineated sections of a voicemail, retaining only those portions which he or 
she desires to save. Additional actions would include, but are not limited to: replaying 
the section, and saving the section. 

Figure 2: Playing a voicemail which has delineation 



Figure 4: Histogram for the GUI; GUt is web driven for example 



Printed for "Marcia L. Doubet" <mld@mindspr ing . com> 



2 



Renee Kovales, 09:39 AM 10/12/20, Fw: Patent info 



Text to Speech - Audio Renderings for Expressing Text Nuances: 

Description of the Problem: 

Face to face communication between people involves many parallel communication paths. We derive information from body language, from 
words, from intonation, from fecial expressions, from the distance between our bodies. Distance communication generally involves only a few 
of these paths, with users trying to overcome the limitations so imposed. Today, unified messaging and network convergence exacerbates 
the problem by adding the difficulties of media transformation to our communication. E-mail tries to overcome these problems by using font 
sizes, colors and emoticons (combination of text symbols which bear a semblance to facial expressions) to express non-text information. 
This information includes emphasis, emotion, irony, etc. When e-mail is transformed via an audio read out, as is done in unified messaging, 
this context can be lost. The loss of the context of messages may result in a loss of understanding of the topic or the underlying meaning of 
the message. 

In addition to the loss of context, translations are sometimes inaccurate. White the recipient listens to the translations, they may not be 
aware of which parts of the translation are accurate and which are not. The recipient must either trust that the information is 1 00% accurate, 
or assume that part or none of it is accurate. In either case, a loss in communications may occur, and certainly there is a potential for a loss 
of time. There is a need to identity to the recipient the certainty of the translation. 



Description of the Invention: 

Our invention, Text to Speech - Audio Renderings for Expressing Text Nuances, is a system and method to provide additional context during 
audio messages. 

Context Enhancement: An audio cue can be used to minimize the effect of a media 
transformation from a non-audio source such as text. In one embodiment, each paragraph 
separation is taken to be a KAS (as described above in Recording and Receiving Voice Mail 
with Freeform Bookmarks) , and an audio cue is mixed into the message. In another embodiment, 
an appropriate audio file is mixed with the resultant audio from the transformation, and 
thereby providing parallel information as to context. In this invention, we are not 
inserting an audio sound or cue in-line in the message (e.g., a giggle sound in place of a 
smiley emoticon) , but we are "conferencing" an appropriate audio file for part of, or all of, 
the message. An appropriate cue may be determined in several ways. If the originating 
source has supplied keywords, this can be used as a source of cueing. If the originating 
source has not supplied keywords , (e . g . , today * s e-mail systems) , the transformation may 
select an appropriate noun in the first few sentences to use as a source of cueing. For 
example, if the first sentence of a paragraph reads "The wedding date has been set.", the 
appropriate audio cue may be church bells. If the sentence reads "The meeting was very 
productive" the appropriate audio cue may be papers rustling, and low conversation. 

Translation Accuracy: Audio cues can provide additional contextual information, especially 
for transformations. For example, when machine language translation is involved, audio cues 
can be used to indicate degree of certainty in the translation. A background hum might 
indicate certainty of translation, with higher pitches indicating more certainty and lower 
pitches indicating less. Another option is to have the pitch of the voice change to 
indicate that the certainty of the translation. Note that this may be usefully employed in 
an audio to audio transform, with voice recognition in the middle, or a text to speech plus 
language transform. Another example would be in the use of speaker identification to 
identfiy the speaker leaving a voice mail. The background tone can indicate the degree of 
certainty in the identification. Similar methods can be used to highlight passages in text 
as to degree of certainty, when for instance the text was obtained through voice recognition. 
Another example, in a text to speech system an audio cue could be used to indicate the color 
of the text (which may indicate the degree of importance, a change in topic, and etc.); the 
background hum or voice as described above could be applied in this case with the pitch 
indicating the level of importance, a different topic, or more simply a change in color. 



Customization may be provided with the use of a stylesheet. Customizations may include the pitch of the tone(s), or the desired indicator. 



Selectable Mixed Background Sound for Voice Messaging System 
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Description of the Problem: 

wn?l t0 f/nm i^T2J UnlC f ti0n f be ^ een Pe ° ple inv , 0lves many paral,el communication paths. We derive information from body language, from 
words, from intonation, from facial expressions, from the distance between our bodies. Distance communication generally involves onlv a few 
of these paths, with users trying to overcome the limitations so imposed. wmmuracanwi generally involves only a few 

Our invention, selectable mixed background sound for voice messaging systems, is a system and method to provide additional context 
aunng audio messages. 

Description of the Invention: 

Audio only messages: In another embodiment, we associate an audio emotional cue with an audio 
message. In this embodiment, the audio file is selected by the speaker to be mixed with the 
message, or inserted in the message directly without transformation. The file selected 
represents the context which the user wishes to convey. For example, for an angry message 
the user may select quiet rolling thunder as a background. The user can select this effect 
before the message, during the message, or after the message is recorded with appropriate 
DTMF tones on the telephone keypad . This can also be done for parts of the message, as 
delineated with KAS tones (see Recording and Receiving Voice Mail with freeform bookmarks) . 

User selected audio background: In this embodiment, the speaker selects the audio background 
as part of the message being left. For example, the speaker may select from a number of 
offered backgrounds (e.g., "top 40") . The music is mixed with the message, as the other 
audio cues above. In an alternate embodiment, the speaker's telephone client receives the 
selection request and communicates it to the voice mail system. If the speaker has unique or 
favorite music (for example as is used to provide ringing tone on a cell phone) , and wishes 
to have this familiar identification (e.g., a speaker's "theme song") as part of the message 
being left, the telephone can communicate the music, or identify the music to the receiving 
voice mail system. 

Note that this embodiment enables a business rnoder of merchandizing play out of popular music, via a subscription, or a pay per use of the 
music. In this model, it is expected that the speaker would pay for the music, as the receiver has no opportunity to accept or decline. This is 
consistent with the Calling Party Pays model standard in telephony. Again, the music may be transmitted from the phone, identified from the 
nhnnfi or suoolied by the voicemail system from a control indicator sent by the phone. 

REDACTED 

Figure 3: Leaving a message with context 



Attachment Converted: " C: \Eudora\attach\LvMsql . PRZ " 

Attachment Converted: " C: \Eudora\attach\playvml . prz " 

Attachment Converted: M C: \Eudora\attach\AudioSegmentsl . PRZ " 

Attachment Converted: " C: \Eudora\attach\msgctxt 6 . prz " 
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Appl. Serial 10/632,177 

Exhibit I 



Renee Kovales, 11:41 AM 10/12/20, *IBM Confidential: Fw: RSW9-20 



To: "Marcia L. Douber" <inldtoindspring.com> 

Subject: *IBM Confidential: Fw: RSW9-2000-0128 background music 
X-Mailer: Lotus Notes Release 7.0 HF242 April 21, 2006 
From: Renee Kovales <rkovales@us.ibm.com> 
Date: Fri, 12 Oct 2007 11:41:01 -0400 

10/i2/2007 k: SeridliZe bY R ° Uter ° n D03NM118/03/M/IBM (Release 8.0|August 02, 2007) at 
09:41:03, 

Serialize complete at 10/12/2007 09:41:03 
X-ELNK- Received- Info: spv=0; 
X-ELNK-AV: 0 

X-ELNK- Info : sbv=0; sbrc=,0; sbf=00; sbw=000; 



Renee Kovales 

Research Triangle Park, NC 27709 
(919) 254-4522, t/1 444-4522 

— Forwarded by Renee Kovales/Raieigh/IBM on 10/12/2007 11:42 AM — 
Marcia Doubet/Raleigh/Contr/IBM 

12/14/2000 05:58 PM 

Renee Kovales/Raleigh/IBM@IBMUS To 

cc 

*IBM Confidential: RSW9-2000-01 28 background music Subject 



Hii IVe finished reading the finat disclosure. |Z D ACTE D 



For selecting an audio background, you have 3 embodiments: 

1) for the entire message 

2) for parts of the message 

3) announcement of the speaker (using his theme); could come at the end, or at the beginning 

REDACTED 

also, there are the choices of select before, during, or after speaking the message 

For how the selecting process works, again 3 embodiments: 

1) transmit the music from the phone 

2) transmit an identification of the music from the phone 

3) voice mail system sends a selection list to the phone, caller selects one & sends it back 



REDACTED 



That's it for now. Ill be on vacation from 12/22 through 12/27 but will be checking my mail at other times. Regards - Marcia 
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Ranee Kovales, 09:42 AM 10/12/20, *IBM Confidential: Pw: Re: RSW 



To: "Marcia L. Doubet" <mldtoindspring.com> 

Subject: *IBM Confidential: Fw: Re: RSW9-2000-0128 background music 
X-Mailer: Lotus Notes Release 7.0 HF242 April 21, 2006 
From: Renee Kovales <rkovales@us * ifam* com> 
Date: Fri, 12 Oct 2007 09:42:49 -0600 

X-MIMETrack: Serialize by Router on D03NM118/03/M/IBM (Release 8.0 (August 02, 2007} at 
10/12/2007 

09:42:50 
X-ELNK-AV: 0 

X~ELNK~Info: sbv=0; sbrc=.0; sbf=00; sbw=000; 



Renee Kovales 

Research Triangle Park, NC 27709 
(919) 254-4522, t/1 444-4522 

— Forwarded by Renee Kovales/Raleigh/IBM on 10/12/2007 1 1:43 AM — 
Renee Kovales/Raleigh/IBM 

12/28/2000 02:50 PM 

To 

Marcia Doubet/Raleigh/Contr/IBM 

Edie Stern/Fort Lauderdale/IBM@IBMUS, Barry Wiliner/Watson/IBM@IBMUS 

Subject 

*IBM Confidential: Re: *IBM Confidential: RSW9-2000-01 28 background musicLink 



Hi Marcia, 

Here are a couple of the charts. These charts show how a caller leaves a msg with a background when the voice mail system is doing ail of 
the work (i.e., the voice mail system prompts for the background and controls the background). A couple of items I'm not sure how to clarify 
in the charts (or if clarification is required): 

* AH of the charts have the words " Message recording starts". The voice mail system is not recording the sound of the DTMF tone 
for the user to hear; it is storing the DTMF tones for further processing. 

* In the data structure I showed the voice mail system storing the voice msg separately from the background. But, the system could 
store the mix of the msg and background (I didnt show this). I think that from a storage standpoint it would be preferable to store the 
msg and the background separately and mix during playback, but that may have some performance implications (I don't know). 

* If the caller does not select a background for a segment, the background selection is none. 

* If the caller presses * to select a new background and has already selected a background, the most recent selection will be stored. 

* On figures 5 and 6, the caller can select a background at any time. Given this, the caller is able to select a background prior to 
speaking, and is not required to speak after the selection. If the caller does not speak at all, then I would think that segment would not 
be stored or if the caller did not leave any message, the voice system would simply store the fact that a msg was left from extension 
xyz (similar to the voice mail system at our site). 



Let me know what you think about these. I'll work on the other charts for this on Friday. Do 1 need a chart showing the playback? 
Renee 



Renee Kovales 

Research Triangle Park, NC 
(919) 254-4522, t/1 444-4522 



Marcia Doubet 

12/14/00 05:58 PM 

To: Renee Kovales/Ra!eigh/IBM@IBMUS 

cc: 
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Renee Kovales, 09:42 AM 10/12/20, *IBM Confidential: Fw: Re: RSW 



From: Marcia Doubet/Raleigh/Contr/lBM@ibmus 
Subject: *IBM Confidential: RSW9-2000-0128 background music 



Hi! I've finished reading the final disclosure. R £ D ACTE D 



For selecting an audio background, you have 3 embodiments: 

1) for the entire message 

2) for parts of the message 

3) announcement of the speaker (using his theme); could come at the end, or at theJaeginning 

REDACTED 

- also, there are the choices of select before, during, or after speaking the message 

For how the selecting process works, again 3 embodiments: 

1 ) transmit the music from the phone 

2) transmit an identification of the music from the phone 

3) voice mail system sends a selection list to the phone, caller selects one & sends it back 



REDACTED 



That's it for now. I'll be on vacation from 12/22 through 12/27 but will be checking my mail at other times. Regards - Marcia 
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Renee Kovales, 09:45 AM 10/12/20 , Fw: RSW9-2000-0128 background 



To: "Marcia L. Doubef <mldgmindspring.com> 
Subject: Fw: RSW9-2000-0128 background music 
X-Mailer: Lotus Notes Release 7.0 HF242 April 21, 2006 
From: Renee Kovales <r kovales @ us . ibm. com> 
Date: Fri, 12 Oct 2007 09:45:16 -0600 

X-MIMETrack: Serialize by Router on D03NM118/03/M/IBM (Release 8.0|August 02, 2007) at 
10/12/2007 

09:45:18 
X-ELNK-AV : 0 

X-ELNK-Info: sbvK); sbrc=.0; sbf=00; sbw-000; 



Renee Kovales 

Research Triangle Park, NC 27709 
(919) 254-4522, t/1 444-4522 

— Forwarded by Renee Kovales/Raleigh/IBM on 10/12/2007 1 1 :46 AM — 
Ranee Kovales /Raleigh / IBM 

12/29/200001:51 PM 

Marcia Doubet^aleigh/Contr/IBM@ibmus T ° 

Edie Stern/Fort Lauderdale/IBM@IBMUS, Barry Willner/Watson/IBM@IBMUS X 

Re: RSW9-2000-0128 background music Subbed 



Hi Marcia, 

The foHowing chart shows a caller leaving a music/audio selection in a message (not background sound). Note that the caller could leave 
one or more selections without any spoken message. Additionally, it is possible to mix this function with the selectable background function 
(sent you some charts on this yesterday). The background and the selected music/audio would not be playing at the same time. To provide 
this combined function the system could have a menu system for these types of services so that when the caller presses * the system 
prompts for "Insert an audio file" or "Select background". And/or the person could press *1 for insert audio file or *2 for select background 
and bypass the first level of menus (ideally they could bypass all levels and key in *2156 for background selection 156). Current voice mail 
systems have * types of functions to listen to the message you just recorded, and etc. I did not do a chart on this but can rf you think we need 
one. Additionally, the code structure that you might use to do this would change as well - 1 put that in this attachment as well. 



I haven 1 1 completed everything you listed in your note, but will send you updates as I do. 
Let me know if you have any questions or comments. 

Renee 

Renee Kovales 

Research Triangle Park, NC 
(919) 254-4522, t/1 444-4522 

Attachment Converted: ~" C : \Eudora\attach\msgctxt22 . prz " 
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Renee Kovales, 09:48 AM 10/12/20, Fw: RSW9-2000-0128 background 



To: "Marcia L. "Doubet'* 1 <mldi5inindspring . com> 
Subject: Fw: RSW9-2000-0128 background music 
X-Mailer: Lotus Notes Release 7.0 HF242 April 21, 2006 
From; Renee Kovales <rkovales@us, ibm, com> 
Date: Fri, 12 Get 2007 09:48:31-0600 

X-MIMETrack: Serialize by Router on DQ3NM118/03/M/IBM (Release 8.0|August 02, 2007) at 
10/12/2007 

09:48:33 
X-ELNK-AV: 0 

X-ELNK-Info: sbv=0; sbrc=.0; sbf=00; sbw=000; 



Renee Kovales 

Research Triangle Park, NC 27709 
(919) 254-4522, t/1 444-4522 

— Forwarded by Renee Kovaies/Raieigh/IBM on 10/12/2007 1 1 :49 AM 

Renee Kovales /Raleigh /IBM 

12/29/2000 04:37 PM 

To 

Marcia Doubet/Raleigh/Contr/IBM@ibmus 

cc 

Edie Stern/Fort l_auderdale/IBM@IBMUS, Barry WiiIner/Watson/IBM@lBMUS 

Subject 

Re: Re: RSW9-2000-0128 background music 



Hi Marcia, 

Here is the next part: 

Transmitting the music from the calling phone: 

The calling phone would need: 

* storage media for the audio selections (could be memory, FLASH, or some other removable storage) 

* Pre-programmed buttons 

* way to download the audio. Could do this over the phone line or could have a connection such as a USB connection, and then hook 
up to the computer to download, or could have removable preprogrammed media. 

* capability for caller to hear the audio as well (this could be optional). 

Here are some charts describing the background selection: 



Note that if the caller wanted to leave an entire audio file, he would have to listen to it as well since the remote side is only recording, f guess 
you could design a phone to keep the connection alive eventhough you've hung up (similar to a hold button on an office phone), play the 
audio file you've selected, and then automatically disconnects when it is done. Not sure if we want to include this. Edie, Barry, any 
thoughts? 



REDACTED 



Renee 

Renee Kovales 

Research Triangle Park, NC 
(919) 254-4522, t/1 444-4522 

Attachment Converted: " C: \Eudora\attach\msgctxt33 . prz " 
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Jim Mathewson II, 09:05 AM 1/8/01 -, Re: RSW9-2000-0126 through -01 



Importance: Normal 

Subject: Re: RSW9-2000-012 6 through -0128 
To: "Marcia L. Doubet" <mld@mindspring. com> 
X-Mailer: Lotus Notes Release 5.0 March 30, 1999 
From: "Jim Mathewson II" < jramii@us . ibm. com> 
Date: Mon, 8 Jan 2001 09:05:09 -0500 

X-MIMETrack: Serialize by Router on D04NM200/04/M/IBM(Release 5.0.3 (Intl)j21 March 2000) 
at 

01/08/2001 09:05:17 AM . . ., 



REDACTED 



"Marcia L. Doubet" <mld@mindspring . cgm> on 01/03/2001 04:16:40 PM 

To: Renee Kovales /Raleigh/ I BM@IBMUS, Edie Stern/Fort 

Lauderdale/IBMQIBMUS, Barry Willner/Watson/IBM@IBMUS , Jim Mathewson 
II/Raleigh/IBM@IBMUS 

cc : 

Subject: RSW9-2000-0126 through -0128 



Hi, Everyone - I will be starting to write up these applications in the 
next couple of days. We have a Jan. 31st due date. My plan is to have a 
first draft of one of them ready to send to you by Monday, Jan. 8th, and 
first drafts of the other 2 to follow during that week. I know that some 
of you will be gone that week, so we'll need to work around your schedules. 

I would appreciate it if you would send me a return e-mail setting out 
your availability during January. 



Thanks very much for all the info you supplied, Renee! Regards - Marcia 



Marcia Doubet 

phone and fax: (919) 557-3827 
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Renee Kovales, 11:49 AM 10/12/20, *IBM Confidential: Fw: RSW9-20 

To: "Marcia L. Dcmbef" <mldtaindspring . com> 

Subject: *IBM Confidential: Fw: RSW9-2000-0128 first draft 

X-Mailer: Lotus Notes Release 7.0 HF242 April 21, 2006 

From: Renee Kovales <rkovales(?us . ibm. com> 

Date: Fri, 12 Oct 2007 11:49:39 -0400 

10/12/2007 k: S " laliZe by R ° Uter ° n D03NM118/03/M/IBM (Release 8.0|August 02, 2007) at 
09:49:41, 

Serialize complete at 10/12/2007 09:49:41 
X-ELNK-Received-Info: spv=0; 
X-ELNK-AV : 0 

X-ELNK-Info: sbv=0; sbrc-.O; sbf=00; sbw-000; 



Renee Kovales 

Research Triangle Park, NC 27709 
(919) 254-4522, t/1 444-4522 

— Forwarded by Renee Kovales/Raleigh/IBM on 10/12/2007 11:51 AM — 
Edie Stern/Fort Lauderdale/ IBM 

01/10/2001 06:37 PM 

Marcia Doubet/Raleigh/Contr/IBM T ° 

cc 

Barry Willner/Watson/IBM, Renee Kovales/Raleigh/IBM 

Re: *IBM Confidential: RSW9-2000-0128 first draftUnk Subject 



Marcia, 

First set of markups (I made it to page 14 or so). Will pick up from there and send more tomorrow. 
Cheers... Edie 

Edith H. Stern 

Manager, Business Integration Technologies, IBM Research 

Member of the IBM Academy of Technology 

Phone: 914 784-7275, TL 863-7275, Fax: 914 784-6032 

To: Renee Kovales/Raleigh/IBM® IBMUS, Edie Stern/Fort Lauderdale/IBM@IBMUS, Barry WHIner/Watson/IBM@IBMUS 

cc: 

From: Marcia Doubet/Raleigh/Contr/IBM@ibmus 
Subject: *IBM Confidential: RSW9-2000-01 28 first draft 



Hi, everyone? Here's the first draft of your apprication on sefectabte sounds in voicemaif. 

REDACTED 
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These inventions are supposed to be finished by Jan. 31 st, as I mentioned earlier. We have pientv of time but we do need to make oood 
progress. So, I'd like to get your input on this no later than Monday, 1/15. .. REDACTED 



Regards - Marcia (919-557-3827) 



#### RSW9-2000-0128EHS.Iwp has been removed from this note on 20 June 2001 by Renee Kovales 



Printed for "Marcia L. Doubet" <mld@mindspring.com> 
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k :>* Marcia Doubet 

01/17/2001 12:36 PM 

To: Edie Stern/Fort Lauderdale/IBM@IBMUS 

cc; Renee Kovales/Raieigh/IBM@IBMUS, Barry Willner/Watson/IBM@IBMUS 

i-rom: Marcia Doubet/Raleigh/Contr/IBM@ibmus 

Subject: *IBM Confidential: Re: RSW9-2000-0128 second draft Qj 

Importance: Normal 



Hi, Edie - You raise some good points. I suggest we 



REDACTED 



Marcia - comment on the first claim 

We claim: 

1 A method of enhancing voice mail messages, comprising steps of: 



REDACTED 
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^ Marcia Doubet 

01/22/2001 03:46 PM 

To: Edie Stern/Fort Lauderdale/IBM@IBMUS 

cc; 

From: Marcia Doubet/Raleigh/Contr/IBM@ibmus 

Subject: Re: claims on 128 El 

Importance: Normal 

Hi, Edie - I plan to send revised claims with the 3rd draft, since I had to move on to the other 
applications for now. How about if you skim through the ones that are there to see if any topics 
are left uncovered that you would like added? I know I forgot to add claims for the phone 
conversation mode. Other than that, I don't know of anything. It will be late tomorrow or Wed. 
before I get back to -0128. Thanks for your input. Regards - Marcia 
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Renee Kovales, 11:47 AM 10/12/20, Fw: RSW9-2000-0128 final draft 

To: "Marcia L. Doubet" <inidtaincispring . com> 

Subject: Fw; RSW9-2000-0128 final draft 

X-Mailer: Lotus Notes Release 7.0 HF242 April 21, 2006 

From; Renee Kovales <rkovales@us . ibiru com> 

Date: Fri, 12 Oct 2007 11:47:05 -0400 

10/12/2007 k : 3erlaliZe bY R ° Uter ° n D03NM118/03/M/IBM (Release 8.0|August 02, 2007) at 
09:47:07, 

Serialize complete at 10/12/2007 09:47:07 
X-ELNK-Received-Inf o : spv=0 ; 
X-ELNK-AV: 0 

X-ELNK-Info: sbv=0; sbrc-.O; sbf^OO; sbw=000; 



Renee Kovales 

Research Triangle Park, NC 27709 
(919) 254-4522, t/1 444-4522 

Forwarded by Renee Kovales/Raleigh/IBM on 10/12/2007 1 1 :37 AM 

Marcia Doubet/Raleigh/Contr/IBM 

01/31/2001 03:19 AM 

Renee Kovates/Raleigh/IBM@IBMUS, Edie Stern/Fort Lauderdale/IBM@!BMUS, Barry Willner/Watson/IBM@IBMUS ' ° 

Jeanine Ray/Raleigh/IBM@IBMUS 00 

RSW9-2000-0128 final draft SubjeCt 



Hi - Attached is the final draft of this application. Changes are limited to pages 50 & 52 (except for the claim renumbering, which applies to 
pages 52 onward and is not marked). I've addressed Edie's final comments as follows: 



REDACTED 



I need a note for my files saying you've approved it. Thanks - Marcia 
[attachment M RSW9-2000-0128.lwp ,, deleted by Renee Kovales/Raleigh/IBM) 
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Law Offices of Marcia L. Doubet 

5228 Westminster Lane 
Fuouay-Varina, NC 27526-9062 
Phone: (9 19)557-3827 

Fax: (9 I 9) 557-3827 
E-Mail: mld@mindspring.com 



January 31, 2001 

Ms. Jeanine Ray-Yarletts, Attorney 
IBM Corporation 
Dept. T81B/Bldg. 503 
P. O. Box 12195 

Research Triangle Park, NC 27709 



RE: 



Patent Application - docket number RSW920000128US1 
Prepared for IBM Attorney 



Jeanine, 

j i h T e compI ? ed the application for the invention titled "Selectable Audio and 
Mixed Background Sound for Voice Messaging System", by Renee Kovales et al and 
having docket number RSW920000128US1. Approval has been received from the 
inventors today. 

w JT1 El ! ClOS n ed iS Q Printed COpy of the A PPlication, and a diskette containing the 

r c t'n *° urce ,™ aterial ^ "rsw9-2000-0128.wpd") and a Freelance drawings file 

REDACTED 

Please ensure that this application is filed on the same day as related aoolications 
RSW920000126US1 and RSW920000127US1. applications 

Best regards, 




Marcia L. Doubet, Esq. 
Registration Number 40,999 



